-
Notifications
You must be signed in to change notification settings - Fork 28.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Minor assembly directory fix in load-spark-env-sh #5261
Conversation
Set the current dir path $FWDIR and same at $ASSEMBLY_DIR1, $ASSEMBLY_DIR2 otherwise $SPARK_HOME cannot be visible from spark-env.sh -- no SPARK_HOME variable is assigned there. I am using the Spark-1.3.0 source code package and I come across with this when trying to start the master: sbin/start-master.sh
Can one of the admins verify this patch? |
(You could give this a more specific title than "Update load-spark-env.sh") I'm not sure about this. For example However CC @vanzin since this used to refer to The lines you reference don't exist in 1.3.0 though. Are you sure you're using 1.3.0? |
Hmm. I see. One comment: doing the following in a script that is sourced is a little weird:
That's because I'd put a comment in the code about that, maybe, or use |
To be more precise, I am running Spark on HPC and I have made the build on scala-2.11; however when I started the master, the error I was receiving was and yes this is on Spark 1.3.0, I have downloaded the package from the following mirror: |
@Raschild what about Marcelo's comment about |
@srowen yes you could use something like @vanzin yes you are right; I only used because it is the same first command in the From what I see it is the same issue in Spark-1.2.0 package...I haven't look at previous versions! |
Yes I was wonder whether it affects |
ok to test |
Test build #29665 has started for PR 5261 at commit |
Test build #29665 has finished for PR 5261 at commit
|
Test FAILed. |
Jenkins, retest this please. |
Test build #29668 has started for PR 5261 at commit |
Test build #29668 has finished for PR 5261 at commit
|
Test PASSed. |
@Raschild this is waiting on your comment and/or change. Can you make the title more descriptive please? |
@srowen |
@Raschild ... hm I would have thought something like I'm still a little confused about which branch this applies to. Yes, I see you're applying this to I'd like to get this resolved one way or the other. |
@srowen - Title Updated |
I've merged this for |
Set the current dir path $FWDIR and same at $ASSEMBLY_DIR1, $ASSEMBLY_DIR2 otherwise $SPARK_HOME cannot be visible from spark-env.sh -- no SPARK_HOME variable is assigned there. I am using the Spark-1.3.0 source code package and I come across with this when trying to start the master: sbin/start-master.sh Author: raschild <raschild@users.noreply.github.com> Closes #5261 from raschild/patch-1 and squashes the following commits: b9babcd [raschild] Update load-spark-env.sh
Set the current dir path $FWDIR and same at $ASSEMBLY_DIR1, $ASSEMBLY_DIR2 otherwise $SPARK_HOME cannot be visible from spark-env.sh -- no SPARK_HOME variable is assigned there. I am using the Spark-1.3.0 source code package and I come across with this when trying to start the master: sbin/start-master.sh Author: raschild <raschild@users.noreply.github.com> Closes #5261 from raschild/patch-1 and squashes the following commits: b9babcd [raschild] Update load-spark-env.sh
Set the current dir path $FWDIR and same at $ASSEMBLY_DIR1, $ASSEMBLY_DIR2
otherwise $SPARK_HOME cannot be visible from spark-env.sh -- no SPARK_HOME variable is assigned there.
I am using the Spark-1.3.0 source code package and I come across with this when trying to start the master: sbin/start-master.sh